Operation receiving apparatus, control method, image forming system, and recording medium

ABSTRACT

An apparatus includes: a display; and a controller that: causes the display to display a plurality of operation items that can be instructed by a user, receives an instruction by a voice operation with respect to at least one of the operation items, and receives an instruction by a manual operation with respect to at least one of the operation items. The display displays a first operation item identifiable from the remaining operation items, and the first operation item can be instructed by a user voice.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation application of U.S. patentapplication Ser. No. 16/823,053, filed on Mar. 18, 2020 and, claimspriority to Japanese Patent Application No. 2019-76943, filed on Apr.15, 2019, both of which are incorporated herein by reference in theirentirety.

BACKGROUND Technical Field

The present invention generally relates to an operation receivingapparatus, a control method, an image forming system, and a recordingmedium.

Description of Related Art

In a print shop and the print specialized department in a large company(hereinafter, a print shop and the like), a plurality of image formingapparatuses are installed. For example, since a large production printer(PP machine) is expensive, the down time is required to be minimized toenhance the investing effect. Thus, an operator in the print shop andthe like is required to minimize the down time of a plurality of PPmachines and perform the complicated printing setting for a short timeto continuously operate the PP machines.

However, when the image forming apparatus is operated, a large number ofcomplicated printing settings are required to be performed to execute ajob. In particular, in the PP machine in which the number of usablepaper types and of post-processing types is large, a large number ofprinting settings are required to be performed to execute the job. Toassist the input of the manual printing setting in performing such thecomplicated printing setting, an image forming apparatus in which theprinting setting is inputted by using voice has been proposed (forexample, see Patent Literature 1 (JP-2007-18090A) and Patent Literature2 (JP-2014-112129A)).

PATENT LITERATURE

Patent Literature 1: JP-2007-18090A

Patent Literature 2: JP-2014-112129A

However, in the image forming apparatus, there is the case where thenumber of functions is large, and there is the case where the number oflong and complicated set values, such as a paper name, is large, it isdifficult to perform the printing setting for a short time even when theinput of the printing setting is assisted by using voice. Thus, anoperation receiving apparatus and an image forming system that canperform the printing setting for a shorter time are required.

One or more embodiments of the present invention provide an operationreceiving apparatus, a control method, an image forming system, and arecording medium that can perform the printing setting for a short time.

SUMMARY

According to one or more embodiments of the present invention, anapparatus includes: a display; and a controller that: causes the displayto display a plurality of operation items that can be instructed by auser, receives an instruction by a voice operation with respect to atleast one of the operation items, and receives an instruction by amanual operation with respect to at least one of the operation items,wherein the display displays a first operation item to be identifiablefrom the remaining operation items, and the first operation item can beinstructed by a user voice.

Also, an image forming system of one or more embodiments of the presentinvention comprises the aforementioned apparatus and an image formingapparatus.

According to one or more embodiments of the present invention, a methodincludes: causing a display to display a plurality of operation itemsthat can be instructed by a user, receiving an instruction by a voiceoperation with respect to at least one of the operation items, andreceiving an instruction by a manual operation with respect to at leastone of the operation items, wherein the display displays a firstoperation item to be identifiable from the remaining operation items,and the first operation item can be instructed by a user voice.

Also, according to one or more embodiments of the present invention, anon-transitory recoding medium storing a computer readable program forcausing a computer to execute: causing a display to display a pluralityof operation items that can be instructed by a user, receiving aninstruction by a voice operation with respect to at least one of theoperation items, and receiving an instruction by a manual operation withrespect to at least one of the operation items, wherein the displaydisplays a first operation item to be identifiable from the remainingoperation items, and the first operation item can be instructed by auser voice.

BRIEF DESCRIPTION OF THE DRAWINGS

The advantages and features provided by one or more embodiments of theinvention will become more fully understood from the detaileddescription given hereinbelow and the appended drawings which are givenby way of illustration only, and thus are not intended as a definitionof the limits of the present invention;

FIG. 1 is a diagram illustrating the configuration of an image formingsystem according to one or more embodiments;

FIG. 2 is a block diagram illustrating the configuration of an imageforming apparatus according to one or more embodiments;

FIG. 3 is a block diagram illustrating the configuration of an operationreceiving apparatus according to one or more embodiments;

FIG. 4 is a block diagram illustrating the configuration of an operationreceiving unit of the image forming apparatus according to one or moreembodiments;

FIG. 5 is a block diagram illustrating the configuration of an externalserver according to one or more embodiments;

FIG. 6 is a block diagram illustrating the configuration of acontrolling unit according to one or more embodiments;

FIG. 7 is a diagram illustrating an initial screen displayed at thestart of the operation by a user according to one or more embodiments;

FIG. 8 is a diagram illustrating the state where a first display regionand a second display region are displayed on a displaying unit accordingto one or more embodiments;

FIG. 9A is a diagram illustrating the state where the first displayregion and the second display region are displayed on the displayingunit according to one or more embodiments;

FIG. 9B is a diagram illustrating the state where a third display regionis singly displayed on the displaying unit according to one or moreembodiments;

FIG. 10A is a diagram illustrating the state where the first displayregion and the second display region are displayed on the displayingunit according to one or more embodiments;

FIG. 10B is a diagram illustrating the state where the third displayregion is singly displayed on the displaying unit according to one ormore embodiments;

FIG. 11A is a diagram illustrating the state where the first displayregion and the second display region are displayed on the displayingunit according to one or more embodiments;

FIG. 11B is a diagram illustrating the state where the third displayregion is singly displayed on the displaying unit according to one ormore embodiments;

FIG. 12A is a diagram illustrating the state where the first displayregion and the second display region are displayed on the displayingunit according to one or more embodiments;

FIG. 12B is a diagram illustrating the state where the third displayregion and the first display region are displayed on the displaying unitaccording to one or more embodiments;

FIG. 12C is a diagram illustrating the state where the initial screen issingly displayed on the displaying unit according to one or moreembodiments; and

FIG. 13 is a flowchart of a control method of operation receptionaccording to one or more embodiments.

DETAILED DESCRIPTION

Embodiments of the present invention will be described below withreference to the drawings, but the scope of the present invention is notlimited to the disclosed embodiments.

Configuration of Image Forming System

FIG. 1 illustrates, as a schematic block diagram of an image formingsystem according to one or more embodiments, a block diagram of theimage forming system including an operation receiving apparatus.

FIG. 1 is a block diagram illustrating an example of the configurationof the image forming system. In an image forming system 1 illustrated inFIG. 1, an operation receiving apparatus 10, an image forming apparatus20, and an external server 30 are communicatively connected via anetwork 2, such as a LAN (Local Area Network). The network 2 may bewired or wireless. For example, given is an example in which theoperation receiving apparatus 10 is connected to the image formingapparatus 20 and the external server 30 via the wireless LAN, and theimage forming apparatus 20 is connected to the external server 30 viathe wired LAN.

The image forming system 1 receives a printing job from, for example, aPC (Personal Computer) and the like and a tablet terminal that are notillustrated, the operation receiving apparatus 10, and the like, whichare connected to the network, and performs various printing according tothe job. The terminal device, such as the PC, includes an applicationcreating a printing document, and a printer driver generating theprinting job.

Configuration of Image Forming Apparatus

Next, FIG. 2 illustrates a block diagram illustrating an example of theconfiguration of the image forming apparatus 20 of the image formingsystem 1. The image forming apparatus 20 illustrated in FIG. 2 includesa displaying unit (or display) 21, an operation receiving unit (or userinterface) 22, an image forming unit 23, a network connecting unit 24, avoice inputting/outputting unit 25, and a controlling unit (orcontroller) 26. These blocks are mutually communicatively connected by asystem bus, not illustrated.

The controlling unit 26 includes an acquiring unit 27 and a displaycontrolling unit 28. The acquiring unit 27 acquires a recognition resultrelated to the voice of a user. The voice recognition related to thevoice of the user is performed by a voice recognizing unit 31 of theexternal server 30 described later (see FIG. 5). And, the acquiring unit27 acquires the voice-related recognition result from the voicerecognizing unit 31 of the external server 30 via the network 2 and thenetwork connecting unit 14.

The display controlling unit 28 is a processing unit controlling ascreen display operation on the displaying unit 21. For example, thedisplay controlling unit 28 displays, on the displaying unit 21, anoperation screen and the like for operating the image forming apparatus20.

The operation receiving unit 22 includes a touch panel and operationkeys. The displaying unit 21 includes a display device. The displayingunit 21 and the operation receiving unit 22 are configured, for example,in such a manner that the display device, such as a liquid crystaldisplay device and the position instructing device of the touch panel ofthe optical type, the electrostatic capacitance type, and the like areoverlapped. The operation receiving unit 22 displays the operationscreen on the display device, and identifies the position instructed bythe user on the operation screen. The controlling unit 26 causes thedisplaying unit 21 to display the operation screen on the basis of thepreviously stored data for screen display. An operation signalindicating the identified instruction position on the operation screen(touched position) or the depressed key is inputted to the controllingunit 26. The controlling unit 26 identifies the operation content fromthe depressed key or the instruction position on the operation screenthat is being displayed, and executes the process on the basis of theoperation content.

The image forming unit 23 performs image formation printing an image onpaper. The voice inputting/outputting unit 25 is a signal convertingunit converting the voice of the user to an electric signal. Further,the voice inputting/outputting unit 25 is a signal converter or signalconverting unit converting an electric signal instructed by thecontrolling unit 26 to voice. The voice inputting/outputting unit 25performs the reception of voice input and voice output by theinstruction of the controlling unit 26. The network connecting unit 24is a communication device or communicating unit performing communicationvia the network 2.

Next, FIG. 3 illustrates a block diagram illustrating an example of theconfiguration of the operation receiving apparatus 10 of the imageforming system 1. The operation receiving apparatus 10 illustrated inFIG. 3 includes a displaying unit 11, an operation receiving unit 12, acontrolling unit 13, a network connecting unit 14, and a voiceinputting/outputting unit 16. The controlling unit 13 includes anacquiring unit 15 and a display controlling unit 17.

The displaying unit 11 of the operation receiving apparatus 10 has thesame configuration as the displaying unit 21 of the image formingapparatus 20, the operation receiving unit 12 of the operation receivingapparatus 10 has the same configuration as the operation receiving unit22 of the image forming apparatus 20, the controlling unit 13 of theoperation receiving apparatus 10 has the same configuration as thecontrolling unit 26 of the image forming apparatus 20, the networkconnecting unit 14 of the operation receiving apparatus 10 has the sameconfiguration as the network connecting unit 24 of the image formingapparatus 20, and the voice inputting/outputting unit 16 of theoperation receiving apparatus 10 has the same configuration as the voiceinputting/outputting unit 25 of the image forming apparatus 20. Also,the acquiring unit 15 of the controlling unit 13 of the operationreceiving apparatus 10 has the same configuration as the acquiring unit27 of the controlling unit 26 of the image forming apparatus 20, and thedisplay controlling unit 17 of the controlling unit 13 of the operationreceiving apparatus 10 has the same configuration as the displaycontrolling unit 28 of the controlling unit 26 of the image formingapparatus 20.

Operation Receiving Unit

FIG. 4 illustrates a block diagram illustrating an example of theconfiguration of the operation receiving unit 22 of the image formingapparatus 20. The operation receiving unit 22 illustrated in FIG. 4includes a touch operating unit 221, a hard key operating unit 222, anda voice operating unit 223. It should be noted that the operationreceiving unit 12 of the operation receiving apparatus 10 can also havethe same configuration as the operation receiving unit 22 of the imageforming apparatus 20 described below.

The touch operating unit 221 generates the operation signal according tothe operation inputted by the position instructing device of the touchpanel of the optical type, the electrostatic capacitance type, and thelike disposed to be overlapped with the displaying unit 21. Further, thetouch operating unit 221 inputs the generated operation signal to thecontrolling unit 26. The hard key operating unit 222 recognizes theoperation inputted to the hard key disposed in the image formingapparatus 20. The voice operating unit 223 generates the operationsignal according to the operation voice inputted from the result of thevoice recognition performed by the voice recognizing unit 31 of theexternal server 30 described later (see FIG. 5). Further, the voiceoperating unit 223 inputs the generated operation signal to thecontrolling unit 26.

Configuration of Server

Next, FIG. 5 illustrates a block diagram illustrating an example of theconfiguration of the external server 30 of the image forming system 1.The external server 30 illustrated in FIG. 5 includes the voicerecognizing unit 31 and a correcting unit 32. It should be noted thatthe operation receiving apparatus 10 or the image forming apparatus 20,not the external server, may have the voice recognizing unit 31 and thecorrecting unit 32.

The voice recognizing unit 31 receives a voice signal inputted to thevoice inputting/outputting unit 25 of the image forming apparatus 20(FIG. 2) via the network 2. And, the voice recognizing unit 31recognizes, from the received voice signal, the voice of the operationinstruction with respect to the image forming system 1. Further, thevoice recognizing unit 31 notifies the controlling unit 13 of theoperation receiving apparatus 10 and the controlling unit 26 of theimage forming apparatus 20 of the recognized operation instructioncontent. The controlling unit 13 of the operation receiving apparatus 10instructs the operation receiving apparatus 10 to execute thepredetermined process according to the operation instruction contentnotified from the voice recognizing unit 31, and the controlling unit 26of the image forming apparatus 20 instructs the image forming apparatus20 to execute the predetermined process according to the operationinstruction content notified from the voice recognizing unit 31.

The correcting unit 32 corrects the information (content) of theinputted voice signal to the operable content closer thereto so that theinputted voice signal matches the operation content that can berecognized by the voice recognizing unit 31. For example, the correctingunit 32 is caused to learn the word that fails in the voice recognition,and on the basis of this learning data, corrects the information of thevoice signal so that it matches the operable content.

The external server 30 can be achieved by a typical computer, such as apersonal computer. Thus, the hardware configuration of the externalserver 30 can be the same as the hardware configuration of the typicalcomputer. Thus, the detailed description of the hardware configurationof the external server 30 is omitted. For example, like the controllingunit 13 described later and illustrated in FIG. 6, the external server30 can be configured of a CPU (Central Processing Unit), a ROM (ReadOnly Memory), a RAM (Random Access Memory), the network connecting unit,and the like.

Controlling Unit

Next, the configurations of the controlling unit 13 of the operationreceiving apparatus 10 and the controlling unit 26 of the image formingapparatus 20 will be described. The controlling unit 13 of the operationreceiving apparatus 10 will be described below as an example, but thecontrolling unit 26 of the image forming apparatus 20 can have the sameconfiguration.

As illustrated in FIG. 6, the controlling unit 13 is configured of a CPU131, a ROM 132, and a RAM 133. The controlling unit 13 is used as anexample of a computer controlling the operation of each unit in theoperation receiving apparatus 10.

The CPU 131 generates the printing job by operating the printer driveraccording to the input operation by the user performed through, forexample, an inputting unit 136. The CPU 131 is an example of thecontrolling unit, and reads the program code of the software achievingeach function according to one or more embodiments from the ROM 132 (anexample of a recording medium) to execute it. It should be noted that inplace of the CPU 131, other calculating devices, such as an MPU (MicroProcessing Unit), may be used as a calculating unit.

The ROM 132 is used as an example of a nonvolatile memory, and storesthe program, data, and the like necessary for the CPU 131 to operate.

The RAM 133 is used as an example of a volatile memory, and temporarilystores the information (data) necessary for each process performed bythe CPU 131.

An outputting unit 135 is configured of, for example, a display device,such as a liquid crystal display monitor, a speaker, and the like. Theoutputting unit 135 corresponds to the function of the displaying unit11 and the voice inputting/outputting unit 16, and outputs the resultand the like of the process performed by the controlling unit 13.

The inputting unit 136 is configured of, for example, a keyboard, amouse, a touch panel, or the like. The inputting unit 136 corresponds tothe function of the operation receiving unit 12 and the voiceinputting/outputting unit 16, the user performs the predeterminedoperation, and the inputting unit 136 receives the input of theoperation instruction.

A nonvolatile storage 134 is an example of a recording material, andstores the program for the CPU 131 to control each unit, the program,such as the OS, and data. As the nonvolatile storage 134, for example,an HDD, an SSD (Solid State Drive), an optical disk, a magneto-opticaldisk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, andthe like are used.

A network I/F (interface) 137 is configured of, for example, an NIC(Network Interface Card), a modem, and the like, establishes theconnection with the device of the communication party via the network 2,such as the LAN, and executes the transmission and reception of variousdata.

Control of Displaying Unit by Controlling Unit

Next, the control of the screen display of the displaying unit 11performed by the controlling unit 13 of the operation receivingapparatus 10 and the control of the screen display of the displayingunit 21 performed by the controlling unit 26 of the image formingapparatus 20 will be described. The control of the screen display of thedisplaying unit 11 performed by the controlling unit 13 of the operationreceiving apparatus 10 will be described below as an example, but thescreen display of the displaying unit 21 performed by the controllingunit 26 of the image forming apparatus 20 can be controlled in the samemanner.

Control-1

FIG. 7 illustrates an example of an initial screen displayed at thestart of the operation by the user on the displaying unit 11 of theoperation receiving apparatus 10. An initial screen 40 illustrated inFIG. 7 displays only a single manual operation screen 51 displaying, inthe display region of the displaying unit 11, an item that can receivean instruction by a manual operation from the user (operation item).

FIG. 8 illustrates the screen display of the displaying unit 11 when theoperation receiving apparatus 10 has acquired, from the voicerecognizing unit 31, the recognition result of the operation instructionwith respect to the image forming system 1 by the voice input by theuser. As illustrated in FIG. 8, the controlling unit 13 of the operationreceiving apparatus 10 performs the control to display, on thedisplaying unit 11, a first display region 41 and a second displayregion 42. The first display region 41 displays an item that can receivean instruction by the manual operation from the user. The second displayregion 42 displays an item that can be instructed by the voice of theuser. In FIG. 8, the manual operation screen 51 displaying the item thatcan be manually operated by the user is displayed in the first displayregion 41 to be reduced in size. And, a voice operation screen 52displaying the item that can be voice instructed by the user isdisplayed in the second display region 42. The second display region 42displays an icon 56 of a microphone so as to indicate that the voiceoperation screen 52 is the screen displayed by the voice input. Also,when the microphone icon 56 is displayed, this indicates that the voiceinput by the voice inputting/outputting unit 16 is enabled.

As illustrated in FIG. 8, the controlling unit 13 displays the manualoperation screen 51 and the voice operation screen 52 on the displayingunit 11 in such a manner that they can be operated in parallel, so thatthe voice operation and the manual operation can be performed inparallel. Thus, the operation receiving apparatus 10 can perform thecomplicated printing setting for a short time.

Also, when the operation intended by the user by the voice operation hasnot been performed on the voice operation screen, and when the voiceoperation screen intended by the user by the voice operation has notbeen displayed, the normal manual operation screen 51 is displayed inparallel with the voice operation screen 52, so that the voice operationand the manual operation can be easily retried.

Control-2

Also, in the state where the first display region 41 and the seconddisplay region 42 are displayed (FIG. 8), and when the manual operationwith respect to the second display region 42 has been received, thecontrolling unit 13 performs the control to display a third displayregion 43 on the displaying unit 11. The third display region 43displays the item that can receive the instruction by the manualoperation from the user. FIGS. 9A and 9B illustrate an example of thescreen display of the displaying unit 11 when the control to display thethird display region 43 on the displaying unit 11 has been performed.

FIG. 9A illustrates the state where the manual operation screen 51 isdisplayed in the first display region 41 and the voice operation screen52 is displayed in the second display region 42. In this state, when theuser has performed the manual operation on the voice operation screen 52in the second display region 42, the controlling unit 13 performs thecontrol to display the third display region 43, as illustrated in FIG.9B. The third display region 43 displays, on the displaying unit 11, theitem that reflects the display content of the second display region 42and can be manually operated by the user.

Also, when as illustrated in FIG. 9B, performing the control to displaya manual operation screen 53 in the third display region 43, thecontrolling unit 13 can perform the control to singly display only thethird display region 43 on the displaying unit 11.

Further, when performing the control to display the manual operationscreen 53 in the third display region 43 illustrated in FIG. 9B, thecontrolling unit 13 can perform the control to receive only the input bythe manual operation with respect to the display content of the thirddisplay region 43. FIG. 9B illustrates that the microphone icon 56 isgray-out displayed and the voice input is not enabled.

It should be noted that in the third display region 43, the microphoneicon 56 for switching the manual operation mode to the voice operationmode can be displayed. By displaying the microphone icon 56, the manualoperation mode can be immediately switched to the voice operation modeeven after the voice operation screen has been changed to the manualoperation screen.

By performing the above control, the input by the manual operation canbe performed with respect to the item displayed on the voice inputscreen after the voice input screen has been started up. For example,when only the start-up of the screen is performed by voice and theoperation thereafter is performed by the manual operation, the voiceoperation screen can be immediately switched to the manual operationscreen. Thus, this is effective when the complicated printing setting inthe operation receiving apparatus 10 is performed for a short time.

Control-3

Also, in the state where the first display region 41 and the seconddisplay region 42 are displayed (FIG. 8), and when the manual operationwith respect to the second display region 42 has been received, thecontrolling unit 13 performs the control to display the third displayregion 43 on the displaying unit 11. The third display region 43displays the item that can receive the instruction by the manualoperation from the user. FIGS. 10A and 10B illustrate an example of thescreen display of the displaying unit 11 when the control to display thethird display region 43 on the displaying unit 11 has been performed.

FIG. 10A illustrates the state where the manual operation screen 51 isdisplayed in the first display region 41 and the voice operation screen52 is displayed in the second display region 42. In this state, when theuser has performed the manual operation on the voice operation screen 52in the second display region 42, the controlling unit 13 performs thecontrol to display a third display region 44, as illustrated in FIG.10B. The third display region 44 displays, on the displaying unit 11,the item that reflects the display content of the first display region41 and can be manually operated by the user.

Also, when as illustrated in FIG. 10B, performing the control to displaya manual operation screen 54 in the third display region 44, thecontrolling unit 13 can perform the control to singly display only thethird display region 44 on the displaying unit 11.

By performing the above control, the voice operation screen can becleared to return to the screen displaying only the manual operationscreen, that is, the initial screen, even after the voice input screenhas been started up. Thus, when the unnecessary voice operation screenhas been displayed in the operation of the operation receiving apparatus10, the voice operation screen can be immediately cleared to displayonly the initial screen.

Control-4

Also, in the state where the first display region 41 and the seconddisplay region 42 are displayed (FIG. 8), and when the manual operationwith respect to the first display region 41 has been received, thecontrolling unit 13 performs the control to display the third displayregion 43 on the displaying unit 11. The third display region 43displays the item that can receive the instruction by the manualoperation from the user. FIGS. 11A and 11B illustrate an example of thescreen display of the displaying unit 11 when the control to display thethird display region 43 on the displaying unit 11 has been performed.

FIG. 11A illustrates the state where the manual operation screen 51 isdisplayed in the first display region 41 and the voice operation screen52 is displayed in the second display region 42. In this state, when theuser has performed the manual operation on the manual operation screen51 in the first display region 41, the controlling unit 13 performs thecontrol to display the third display region 44, as illustrated in FIG.11B. The third display region 44 displays, on the displaying unit 11,the item that reflects the display content of the first display region41 and can be manually operated by the user.

Also, when as illustrated in FIG. 11B, performing the control to displaythe manual operation screen 54 in the third display region 44, thecontrolling unit 13 can perform the control to singly display only thethird display region 44 on the displaying unit 11.

By performing the above control, the voice operation screen can becleared to return to the screen displaying only the manual operationscreen, that is, the initial screen, even after the voice input screenhas been started up. Thus, when the unnecessary voice operation screenhas been displayed in the operation of the operation receiving apparatus10, the voice operation screen can be immediately cleared to displayonly the initial screen.

Control-5

Also, in the state where the first display region 41 and the seconddisplay region 42 are displayed (FIG. 8), and when the voice operationor the manual operation with respect to the second display region 42 hasbeen received, the controlling unit 13 performs the control to voiceoutput the content of the received operation or the control to displaythe content of the received operation on the displaying unit 11. Thecontrolling unit 13 may perform both of the control for the voice outputand the control for the display on the displaying unit 11, or one of thecontrols. FIGS. 12A, 12B, and 12C illustrate an example of the screendisplay of the displaying unit 11 when the control for the voice outputand the control for the display on the displaying unit 11 have beenperformed.

FIG. 12A illustrates the state where the manual operation screen 51 isdisplayed in the first display region 41 and the voice operation screen52 is displayed in the second display region 42. In this state, when theuser has performed the manual operation or the voice operation on thevoice operation screen 52 in the second display region 42, thecontrolling unit 13 performs the control to display a third displayregion 45, as illustrated in FIG. 12B. The third display region 45displays a voice operation screen 55 displaying the operation contentwith respect to the second display region 42 on the displaying unit 11.Further, the controlling unit 13 performs the control to voice outputthe operation content with respect to the second display region 42 onthe displaying unit 11 from the voice inputting/outputting unit 16 ofthe operation receiving apparatus 10 with the control to display thevoice operation screen 55 in the third display region 45.

Also, when the voice operation or the manual operation with respect tothe second display region 42 has been received and a set value withrespect to the setting item has been determined, the controlling unit 13performs the control to display the third display region 45, asillustrated in FIG. 12B. The third display region 45 displays thedetermined set value. Further, the controlling unit 13 performs thecontrol to voice output the determined set value with the control todisplay the determined set value in the third display region 45.

Further, after the set value has been determined, the controlling unit13 performs the control to stop the display of the third display region45. With this, as illustrated in FIG. 12C, the controlling unit 13performs the control to return to the initial screen 40 displaying onlythe single manual operation screen 51 (the same screen as FIG. 7) in thedisplay region for the display content on the displaying unit 11.

Also, when the manual operation with respect to the manual operationscreen 51 in the first display region 41 has been received during thecontrol of the voice output illustrated in FIG. 12B, the controllingunit 13 can perform the control to stop the voice output. In this case,after stopping the voice output, the controlling unit 13 performs thecontrol to return to the initial screen 40 displaying only the singlemanual operation screen 51 in the display region for the display contenton the displaying unit 11, as illustrated in FIG. 12C.

As described above, when the user has performed the manual operation orthe voice operation on the voice operation screen 52 in the seconddisplay region 42, the controlling unit 13 displays the operationcontent in the third display region 45, and voice outputs the operationcontent. With this, the user can check the operation content.

Also, when the manual operation has been performed during the voiceoutput, the voice output is stopped to display the manual operationscreen, so that the next setting operation can be immediately performed.Further, when the set value has been determined, the control to returnthe display screen to the initial screen is performed, so that the nextsetting operation can be immediately performed.

Flowchart of Control Method of Operation Reception

Next, FIG. 13 illustrates a flowchart of an operation example when thecontrolling unit 13 of the operation receiving apparatus 10 performs thecontrol of the screen display of the displaying unit 11 and an operationexample when the controlling unit 26 of the image forming apparatus 20performs the control of the screen display of the displaying unit 21.The process of the flowchart illustrated in FIG. 13 is executed when thecontrolling unit 13 of the operation receiving apparatus 10 or thecontrolling unit 26 of the image forming apparatus 20 has received theoperation by the user. The control of the screen display of thedisplaying unit 11 by the controlling unit 13 of the operation receivingapparatus 10 will be described below as an example. The process of theflowchart illustrated in FIG. 13 is achieved in such a manner that theCPU 131 of the operation receiving apparatus 10 executes the programstored in the ROM 132. Likewise, the screen display of the displayingunit 21 by the controlling unit 26 of the image forming apparatus 20 isalso achieved.

First, the controlling unit 13 displays the initial screen 40 on thedisplaying unit 11, as illustrated in FIG. 7, with the start of theoperation of the operation receiving apparatus 10 by the user (step S1).

Next, the operation receiving apparatus 10 determines whether therecognition result of the voice input by the user by the voicerecognizing unit 31 has been acquired from the external server 30 (stepS2). When the operation receiving apparatus 10 has not acquired therecognition result of the voice input from the external server 30 (NO instep S2), the controlling unit 13 continues the display of the initialscreen 40 until it acquires the recognition result.

When the operation receiving apparatus 10 has acquired the recognitionresult of the voice input from the external server 30 (YES in step S2),the controlling unit 13 displays, in parallel on the displaying unit 11,the first display region 41 that can receive the instruction by themanual operation from the user and the second display region 42displaying the item that can be instructed by the voice of the user(step S3), as illustrated in FIG. 8.

Next, the controlling unit 13 of the operation receiving apparatus 10determines whether the operation receiving apparatus 10 has acquired therecognition result of the voice operation by the user from the externalserver 30 (step S4).

When the operation receiving apparatus 10 has acquired the recognitionresult of the voice operation by the user from the external server 30(YES in step S4), the controlling unit 13 receives the voice operationby the user (step S5). And, as illustrated in FIG. 12B, the controllingunit 13 performs the control to display, in the third display region 45,the voice operation screen 55 displaying the operation content withrespect to the second display region 42 of the displaying unit 11 (stepS6).

Next, the controlling unit 13 of the operation receiving apparatus 10determines whether the set value has been determined with respect to thedisplay content of the third display region 45 by the manual operationor the voice operation by the user (step S7). When the set value has notbeen determined (NO in step S7), the controlling unit 13 continues thedisplay of the third display region 45 until the set value isdetermined.

On the other hand, when the operation receiving apparatus 10 has notacquired the recognition result of the voice operation by the user fromthe external server 30 (NO in step S4), the controlling unit 13 of theoperation receiving apparatus 10 determines whether the manual operationwith respect to the second display region 42 has been performed (stepS8).

When the manual operation with respect to the second display region 42has been performed (YES in step S8), the controlling unit 13 displaysthe third display region 43 on the displaying unit 11 (step S9). Thecontrolling unit 13 displays, on the displaying unit 11, the thirddisplay region 43 including the manual operation screen 53 illustratedin FIG. 9B and displaying the item that reflects the display content ofthe second display region 42 and can be manually operated by the user.Also, as illustrated in FIG. 10B, the controlling unit 13 may display,on the displaying unit 11, the third display region 44 including themanual operation screen 54 displaying the item that reflects the displaycontent of the first display region 41 and can be manually operated bythe user.

Next, the controlling unit 13 of the operation receiving apparatus 10determines whether the set value has been determined with respect to thedisplay content of the third display region 43 or 44 by the manualoperation by the user (step S10). When the set value has not beendetermined (NO in step S10), the display of the third display region 43or 44 is continued until the set value is determined.

When the manual operation with respect to the second display region 42has not been performed (NO in step S8), the controlling unit 13 of theoperation receiving apparatus 10 determines whether the manual operationwith respect to the first display region 41 has been performed (stepS11). When the manual operation with respect to the first display region41 has not been performed (NO in step S11), the process for determiningwhether the recognition result of the voice operation by the user hasbeen acquired from the external server 30 (step S4) is repeated.

When the manual operation with respect to the first display region 41has been performed (YES in step S11), the controlling unit 13 displaysthe third display region 44 on the displaying unit 11 (step S12). Atthis time, as illustrated in FIG. 11B, the controlling unit 13 cansingly display only the third display region 44 on the displaying unit11.

Next, the controlling unit 13 of the operation receiving apparatus 10determines whether the set value has been determined with respect to thedisplay content of the first display region 41 by the manual operationby the user (step S13). When the set value has not been determined (NOin step S13), the controlling unit 13 continues the display of the firstdisplay region 41 until the set value is determined.

When the set value has been determined with respect to the displaycontent of the third display region 45 (YES in step S7), when the setvalue has been determined with respect to the display content of thethird display region 43 or 44 (YES in step S10), or when the set valuehas been determined with respect to the display content of the firstdisplay region 41 (YES in step S13), the controlling unit 13 performsthe control to display the determined set value on the displaying unit11 and the control to voice output the determined set value from thevoice inputting/outputting unit 16 of the operation receiving apparatus10 (step S14).

Further, the controlling unit 13 determines whether the manual operationhas been performed during the voice output in step S14 (step S15). Whenthe manual operation has been performed during the voice output (YES instep S15), the controlling unit 13 performs the control to stop thevoice output (step S16). At this time, as illustrated in FIG. 12C, thecontrolling unit 13 may perform the control to display the initialscreen 40 with the stop of the voice output.

When the manual operation has not been performed during the voice output(NO in step S15), and after the control to stop the voice output in stepS16 has been performed, the controlling unit 13 of the operationreceiving apparatus 10 receives, as the operation instruction content,the determined set value, and executes the setting of the determined setvalue (step S17). After executing the setting of the set value, thecontrolling unit 13 returns to step S1 displaying the initial screen 40on the displaying unit 11, and ends the process by this flowchart.

According to one or more embodiments of the present invention, it ispossible to provide the operation receiving apparatus, the controlmethod, the image forming system, and the recording medium that canperform the printing setting for a short time.

Although the disclosure has been described with respect to only alimited number of embodiments, those skilled in the art, having benefitof this disclosure, will appreciate that various other embodiments maybe devised without departing from the scope of the present invention.Accordingly, the scope of the invention should be limited only by theattached claims.

REFERENCE SIGNS LIST

1 image forming system, 10 operation receiving apparatus, 11, 21displaying unit, 12, 22 operation receiving unit, 13, 26 controllingunit, 14, 24 network connecting unit, 15 acquiring unit, 16, 25 voiceinputting/outputting unit, 2 network, 20 image forming apparatus, 23image forming unit, 30 external server, 31 voice recognizing unit, 32correcting unit, 40 initial screen, 41 first display region, 42 seconddisplay region, 43, 44, 45 third display region, 51, 53, 54 manualoperation screen, 52, 55 voice operation screen, 56 icon, 131 CPU, 132ROM, 133 RAM, 134 nonvolatile storage, 221 touch operating unit, 222hard key operating unit, 223 voice operating unit

What is claimed is:
 1. An apparatus comprising: a display; and acontroller that: causes the display to display a plurality of operationitems that can be instructed by a user, receives an instruction by avoice operation with respect to at least one of the operation items, andreceives an instruction by a manual operation with respect to at leastone of the operation items, wherein the display displays a firstoperation item to be identifiable from the remaining operation items,and the first operation item can be instructed by a user voice.
 2. Theapparatus according to claim 1, wherein the controller receives, withrespect to a second operation item among the operation items displayedon the display, the instruction by the manual operation on a displayscreen of the display.
 3. The apparatus according to claim 1, whereinthe controller receives, with respect to the first operation item, theinstruction by the manual operation.
 4. The apparatus according to claim3, wherein in a state where the first operation item and the secondoperation item are displayed on the display, the controller switches thedisplay from a first state to a second state based on an instruction bythe user, in the first state, the instruction by the manual operationcannot be performed with respect to the first operation item, in thesecond state, the instruction by the manual operation can be performedwith respect to the first operation item, and the controller receives,with respect to the first operation item, the instruction by the manualoperation after the first state has been switched to the second state.5. The apparatus according to claim 1, wherein the first operation itemis displayed in a first region of a display region of the display, andthe second operation item is displayed in a second region of the displayregion of the display.
 6. A non-transitory recoding medium storing acomputer readable program for causing a computer to execute: causing adisplay to display a plurality of operation items that can be instructedby a user, receiving an instruction by a voice operation with respect toat least one of the operation items, and receiving an instruction by amanual operation with respect to at least one of the operation items,wherein the display displays a first operation item to be identifiablefrom the remaining operation items, and the first operation item can beinstructed by a user voice.
 7. The non-transitory recoding mediumaccording to claim 6, wherein the computer further executes: receiving,with respect to a second operation item among the operation itemsdisplayed on the display, the instruction by the manual operation on adisplay screen of the display.
 8. The non-transitory recoding mediumaccording to claim 6, wherein the computer further executes: receiving,with respect to the first operation item, the instruction by the manualoperation.
 9. The non-transitory recoding medium according to claim 6,wherein in a state where the first operation item and the secondoperation item are displayed on the display, the computer furtherexecutes: switching the display from a first state to a second statebased on an instruction by the user, in the first state, the instructionby the manual operation cannot be performed with respect to the firstoperation item, in the second state, the instruction by the manualoperation can be performed with respect to the first operation item, andthe computer further executes: receiving, with respect to the firstoperation item, the instruction by the manual operation after the firststate has been switched to the second state.
 10. The non-transitoryrecoding medium according to claim 6, wherein the first operation itemis displayed in a first region of a display region of the display, andthe second operation item is displayed in a second region of the displayregion of the display.